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(54) Picture feature extraction device, picture retrieving device, attdmetho^ 
feature extraction and retrieving picture 

(57) tn a picture feature extraction device, picture 
data having been determined as a moving picture by a 
picture data identifying section is subjected to being 
taken out the data corresponding to the entire frames or 
a part of the frames by a feature data extracting frame 
selecting means, upon which a resized picture generat- 
ing means generates a resized picture from the data of 
each frame, and a frequency analyzing means analyzes 
a frequency of the resized picture having been gener- 
ated. Then a DC component/partial AC component 
extracting means takes out DC components and a part 
of AC components, after which a feature extraction 
means generates feature data of the whole frames hav- 
ing been selected. The picture data identified as still pic- 
ture data is subjected to having a resized picture 
generated by the resized picture generating means from 
the data of each frame, after which the frequency ana- 
lyzing means executes a frequency analysis on the 
resized picture having being generated- Then as the DC 
compon ent/^artial AC component extracting means 
takes out DC components and a part of AC components 
obtained as a result of the analysis in order to generate 
a feature data. 
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Description 

BACKGROUND OF THE INVENTION 

[OOOI] The present invention relates to a picture 
feature extraction device, picture retrieving device, as 
well as a method of picture feature extraction, and a 
method of retrieving pictures. 

Description of the Related Art 

[0002] As to the conventional picture feature extrac- 
tion device, picture retrieving device and methods 
thereof, it Is typical that they are used in retrieving simi- 
lar pictures or sections of moving pictures from the pic- 
tures being stored. 

[00031 ^11 picture has been often described using 
color feature. Both a color histogram and dominant 
colors aro typically used as picture feature data for 
retrieval. Layout is also a useful feature and can be rep- 
resented using the combination of grid structure, which 
divides a still picture into blocks and uses a color histo- 
gram and local dominant colore for each block- One 
exanpte of a prior art picture retrieving method using a 
reference picture is disclosed in Japanese Patent Laid- 
Open Publication No. 8-249349. In accordance with the 
disclosed retrieving method, the method divides a still 
picture into blocks and uses dominant colors of each 
block as feature data. However, as they need a relatively 
large number of bits for their representation, the similar- 
ity evaluation process requires a large amount of calcu- 
lation. Therefore, this Wnd of combination is not very 
suitable especially for video segment retrieval* which 
requests many repetition of similarity calculation. One 
example of a prior art video segment retrieval method is 
disclosed in United States Patent 5805746. The dis- 
closed retrieving method reduces the calculation cost 
using the combination of grid structure and local aver- 
age colors for high speed detection of only long and 
identical video segment, like TV commercialB, As this 
description represents the color layout roughly, this 
method cannot retrieve still picture and short sections 
with sufficient accuracy. , 
[0004] The following problems are recognized in the 
above-mentioned prior art methods. 
[0005] First in the conventional methods, the fea- 
ture data becomes large, and therefore, it requires a 
large-scale storage unit for storing the feature data. 
Second, a speed required in retrieving decreases 
because the feature data is large. Third, since the 
matching is conducted between the entire inputted mov- 
ing picture feature data and each point of time of the 
moving picture data within the data base subjected to 
retrieval, the time to be consumed for the matching 
process becomes enormous. Particularly, it is a notable 
problem that the data base becomes larger in scale. 
Fourth, if the feature data represents the color layout of 
pictures roughly, still pictures and short sections cannot 
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be retrieved with sufficient accuracy. 
SUMMARY OF THE INVENTION 

[0006] It is therefore an object of the present inven- 
tion to provide a picture feature extraction device, pic- 
ture retrieving device, and methods thereof for 
generating picture feature data and retrieving pictures, 
respectively, capable of retrieving similar still and mov- 
ing pictures from stiU and moving picture data precisely 
and at high speed, so as to generate still picture feature 
data and moving picture feature data requiring leas 
memory capacity. 

[0007] In accordance with a first aspect of the 
present invention, there is provided a picture feature 
extraction device comprising: a resized picture generat- 
ing means generating a resized picture from still picture 
data; a frequency analyzing means executing a fre- 
quency analysis on said resized picture having been 
generated by said resized picture generating means; 
and a DC/partial AC component extracting means tak- 
ing out DC components and a part of AC components 
obtained as a result of the analysis by said frequency 
analyzing means as a picture feature data. 
[0008] In accordance with a second aspect of the 
present Invention, there Is provided a picture feature 
extraction device comprising: a still picture data base 
including a still picture data storing means storing said 
stitl picture data, and a still picture feature data storing 
means storing said still picture feature data generated 
for each still picture data stored in said still picture data 
storing means. ^ t x . 

[0009] In accordance with a third aspect of the 
present invention, there is provided a picture feature 
extraction device comprising: a feature data extracting 
frame selecting means taking out some data corre- 
sponding to the entire frames or a portion of the frames 
from the moving picture data, selecting the data sub- 
jected to processing: a resized picture generating 
means generating a resized picture from data corre- 
sponding to each frame having been taken out; a fre- 
quency analyzing means executing a frequency 
analysis on said resized picture having been generated 
by said resized picture generating means; a DC/partial 
AC component extracting means taking out DC compo- 
nents and a part of AC components obtained as a result 
of the analysis by said frequency analyzing means as a 
frame feature data; and a moving picture feature data 
producing means producing a moving picture feature 
data by collecting said frame feature data outputted 
from said DC/partial AC component extracting means. 
[0010] In accordance with a fourth aspect of the 
present invention, there is provided a picture feature 
extraction device comprising: a moving picture data 
base including a moving picture data storing means 
storing said moving picture data, and a moving picture 
feature data storing means storing said moving picture 
feature data generated for each moving picture data 
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stored in said moving picture data storing means. 
[0011] In accordance with a fifth aspect of the 
present invention, there is provided a picture feature 
extraction device comprising: a picture data identifying 
means identifying whether the inputted picture data is of 
still pictures or of moving pictures; a still picture feature 
extraction means including a resized picture generating 
means generating a resized picture Irom the picture 
data identified as stfll picture data by said picture data 
identifying means, a frequency analyzing means exe- 
cuting a frequency analysis on said resized picture hav- 
ing been generated by said resized picture generating 
means, and a DC/partial AC component extracting 
means taking out DC components and a part of AC 
components obtained as a result of the analysis by said 
frequency analyzing means as a still picture feature 
data; and a moving picture feature extraction means 
including a feature data extracting frame selecting 
moans taking out some data corresponding to the entire 
frames or a portion of the frames from the picture data 
identified as moving picture data by said picture data 
identifying means, selecting the data subjected to 
processing, a resized p'cture generating means gener- 
ating a resized picture from data corresponding to each 
frame having been taken out, a frequency analyzing 
means executing a frequency analysis on said resized 
picture having been generated, a DQwrtraf AC compos 
nent extracting means taking out DC aoniporiants and a 
part of AC components obtained as a result* of the anal- 
ysis by said frequency analyzing means as a frame fea- 
ture data, and a moving picture feature data producing 
means generating a moving picture feature data by col- 
lecting said frame feature data outputted from said 
DC/partial AC component extracting means. 
[0012] , In accordance with a sixth aspect of the 
present invention, there is provided a picture feature 
extraction device comprising; a picture data base 
including a picture data storing means storing said pic- 
ture data, and a picture feature data storing means 
using each p'cture data stored in said picture data stor- 
ing means for storing the still picture feature data gener- 
ated by said still picture feature extraction means and 
the moving picture feature data generated by said mov- 
ing picture feature extraction means as a p'cture feature 
data, 

[0013] In accordance with a seventh aspect of the 
present invention, there is provided a picture feature 
extraction device wherein said resized picture generat- 
ing means divides a still picture into blocks, and takes 
out color data representative of each block from each 
block. 

[0014] In accordance with an eighth aspect of the 
present invention, there is provided a picture feature 
extraction device wherein in case when said picture 
data is compressed by blocks, said resized picture gen- 
erating means takes out DC components from each 
block 

[0015] In accordance with a ninth aspect of the 



present invention, there is provided a picture retrieving 
device comprising: a stfll picture generating means 
including a resized picture generating means generat- 
ing a resized picture from still picture data, a frequency 

s analyzing means executing a frequency analysis on 
safcJ resized picture having been generated by said 
resized picture generating means, and a DOpartial AC 
component extracting means taking out DC compo- 
nents and a pan of AC components obtained as a result 

w of the analysis by said frequency analyzing means as a 
picture feature data; a still picture data base including a 
still picture feature data storing means storing the still 
picture feature data previously being generated by said 
still picture feature extraction means on the basis of the 
still picture data, and a still picture data storing means 
storing said still picture data; and a still picture feature 
data correlation measuring means calculating a correla- 
tion value between the still picture feature data gener- 
ated by said still picture feature extraction means on the 

20 basis of the inputted still picture data and the still picture 
featuredata stored in said still picture data base, the still 
picture feature data correlation measuring means 
retrieving the still picture within the still picture data 
base having a high correlation value as a candidate of 

25. similar still picture. 

BKHBt in acxxirdawce* wrtrr a: tenth* aspect of th> 
present irtventiorr, there* is provided: a pictum retrieving 
device comprising: a moving picture feature etfraciiorr 
means inducing a feature data extracting frame select- 

so ing means taking out some data corresponding to the 
entire frames or a portion of the frames from the moving 
picture data, selecting the data subjected to processing, 
a resized picture generating means generating a 
resized picture from data corresponding to each frame 

w having been taken out a frequency analyzing means 
executing a frequency analysis on said resized picture 
having been generated by said resized picture generat- 
ing means, a DC/partial AC component extracting 
means taking out DC components and a part of AC 

*o components obtained as a result of the analysis by said 
frequency analyzing means as a frame feature data, 
and a moving picture feature data producing means 
producing a moving picture feature data by collecting 
said frame feature data outputted from said DC/partial 
AC component extracting means; a moving p'cture data 
base including a moving picture feature data storing 
means storing the moving picture feature data previ- 
ously being generated by said moving picture feature 
extraction means on the basis of the moving picture 

$o data, and a moving picture data storing means storing 
said moving picture data; and a moving picture feature 
data correlation measuring means calculating a correla- 
tion value between the moving p'cture feature data gen- 
erated by said moving picture feature extraction means 

55 on the basis of the inputted moving picture data and the 
moving picture feature data stored in said moving pic- 
ture data base, the moving picture feature data correla- 
tion measuring means retrieving the moving picture 
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segment within the moving picture data base having a 
high correlation value as a candidate of similar moving 
picture segment 

[00171 In accordance with an eleventh aspect of the 
present invention, there is provided a picture retrieving 
device comprising: a picture data identifying means 
identifying whether the Inputted picture data is of still 
pictures or of moving pictures: a P icture iealure 
extraction means Including a resized picture generating 
means generating a resized picture from the picture 
data identified as still picture data by said picture data 
identifying means, a frequency analyzing means exe- 
cuting a frequency analysis on said resized picture hav- 
ing been generated by said resized picture generating 
means, and a DC/partial AC component extracting 
means taking out DC components end a part of AO 
components obtained as a result of the analysis by said 
frequency analyzing means as a still picture feature 
data: a moving picture feature extraction means includ- 
ing a feature data extracting frame selecting means tak- 
ing out some data corresponding to the entire fram^ or 
a oortion of the frames from the picture data identified 
as moving picture data by said picture data identifying 
means, selecting the data subjected to processing, a 
resized picture generating means generating a resized 
picture from data corresponding to each frame having 
been taken out a frequency analyzing means executing 
a frequency analysis on said resized picture having 
been generated by said resized picture generating 
means, a DC/partial AC component extracting means 
taking out DC components and a part of AC compo- 
nents obtained as a result of the analysis by said fre- 
quency analyzing means as a frame feature data, and a 
moving picture feature data producing means generat- 
ing a moving picture feature data by collechng said 
frame feature data outputted from said DC/partial AC 
component extracting means; a picture data base 
including a picture feature data storing means storing 
the still picture feature data previously being generated 
by said still picture feature extraction means on the 
basis of the still picture data and the moving picture fea- 
ture data previously being generated by said moving 
picture feature extraction means on the basis of the 
moving picture data, and a picture data storing means, 
storing said still picture data and said moving picture 
data: a still picture feature data correlation measuring 
means calculating a correlation value between the still 
picture feature data generated by said still *»; 
ture extraction means on the basis of the inputted still 
picture data and the picture feature data stored In said 
picture data base; and a moving picture feature data 
correlation measuring means calculating a correlation 
value between the moving picture feature data gener- 
ated by said moving picture feature extraction means on 
the basis ol the Inputted moving picture data and the 
picture feature data stored in said picture data base, 
said still picture feature data correlation measuring 
means retrieving the still picture within the picture data 



base having a high con-elation value as a candidate of 
similar still picture, said moving picture feature data cor- 
relation measuring means retrieving the moving picture 
segment within the picture data base having a high cor- 
relation value as a candidate of similar moving picture 
segment 

[0018] In accordance with a twelfth aspect erf the 
present invention, there is provided a picture retrieving 
device wherein said moving picture feature data correla- 
tion measuring means divides the moving picture fea- 
ture data having been generated by said moving picture 
feature extraction means on the basis of said inputted 
moving picture data, and calculates in turn a partial cor- 
relation value between a part of the moving picture fea- 
ture data of the moving picture included in said moving 
picture data base and a moving picture feature data of 
each division of the divided moving picture feature data, 
upon which eliminating the moving picture segment 
included in said moving picture data base from the can- 
didates of similar moving pictures when it has a feature 
data with a partial correlation value lower than a prede- 
termined threshold value. 

fool 91 In accordance with a thirteenth aspect of tne 
present invention, there is provided a picture retrieving 
device wherein said moving picture feature data correla- 
tion measuring means divides the moving picture fea- 
ture data having been generated by said moving picture 
feature extraction means in respect with said inputwd 
moving picture data, and calculates in turn a partial cor- 
relation value between a part of the moving picture fea- 
ture data of the moving picture included in said moving 
picture data base and a partial moving picture feature 
data of each division of the divided moving picture fea- 
ture data, upon which eliminating the moving picture 
segment included in said moving picture data base from 
the candidates of similar moving pictures at the time 
when it is acknowledged that a number of partial moving 
picture feature data having a feature data with a partial 
correlation value lower than a predetermined threshold 
value surpasses a predetermined threshold value. 
[0020] In accordance with a fourteenth aspect of 
the present invention, there is provided a picture retriev- 
ing device wherein said resized picture generabng 
means divides a picture into blocks, and takes out from 
each block color information representative of each 

block. . 
[0021] In accordance with a fifteenth aspect of the 
present invention, there Is provided a picture retrieving 
device wherein said resized picture generating means 
takes out DC components from said each block when 
said picture data is compressed by blocks. 
[00221 In accordance with a sixteenth aspect of the 
present Invention, there is provided a picture feature 
extraction method comprising: a resized picture gener- 
ating step tor generating a resized picture from still pic- 
ture data; a frequency analyzing step for executing a 
frequency analysis on said resized picture having been 
generated by said, resized picture generating step: and 
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a DCfcartial AC component extracting step for taking 
out DC components and a part of AC components 
obtained as a result of the analysis at said frequency 
analyzing step as a picture feature data, 
[0023] In accordance with a seventeenth aspect of 
the present invention, there is provided a picture feature 
extraction method comprising: a still picture data storing 
step for storing said still picture data: and a still picture 
feature data storing step for storing said still picture fea- 
ture data generated for each still picture data stored at 
said still picture data storing step. 
[0024] in accordance with an eighteenth aspect of 
the present Invention, there is provided a picture feature 
extraction method comprising: a feature data extracting 
frame selecting step for taking out some data corre- 
sponding to the entire frames or a portion of the frames 
from the moving picture data, selecting the data sub- 
jected to processing; a resized picture generating step 
for generating a resized picture from data correspond- 
ing to each frame having been taken out; a frequency 
analysing step for executing a frequency analysis on 
said resized picture having been generated at said 
resized picture generating step: a DCfcartia! AC compo- 
nent extracting step for taking out DC components and 
a part of AC components obtained as a result of the 
analysis at said frequency analyzing step as a frame 
feature data: and a moving picture feature data produc- 
ing step for producing a moving picture feature data by 
collecting said frame feature data outputted from said 
DC/partial AC component extracting step. 
[0025] In accordance with a nineteenth aspect of 
the present invention, there is provided a picture feature 
extraction method comprising: a moving picture data 
storing step for storing said moving picture data; and a 
moving picture feature data storing step for storing said 
moving picture feature data generated for each moving 
picture data stored at said moving picture data storing 
step. 

[0026J in accordance with a twentieth aspect of the 
present invention, there is provided a picture feature 
extraction method comprising: a picture data identifying 
step for identifying whether the inputted picture data Is 
of stiJJ pictures or of moving pictures; a stil! picture fea- 
ture extraction step including a resized picture generat- 
ing step for generating a resized picture from the picture 
data identified as still picture data at said picture data 
identifying step, a frequency analyzing step for execut- 
ing a frequency analysis on said resized picture having 
been generated at said resized picture generating step, 
and a DC/partial AC component extracting step for tak- 
ing out DC components and a part of AC components 
obtained as a result of the analysis at said frequency 
analyzing step as a still picture feature data: and a mov- 
ing picture feature extraction step including a feature 
data extracting frame selecting step for taking out some 
data corresponding to the entire frames or a portion of 
the frames from the picture data identif ied as moving 
picture data at said picture data identifying step, select- 



ing the data subjected to processing, a resized picture 
generating step for generating a resized picture from 
data corresponding to each frame having been taken 
out. a frequency analyzing step for executing a fre- 
s quency analysis on said resized picture having been 
generated at said resized picture generating step, a 
DC/|3artial AC component extracting step for taking out 
DC components and a part of AC components obtained 
as a result of the analysis at said frequency analyzing 
to step as a frame feature data, and a moving picture fea- 
ture data producing step for generating a moWng picture 
feature data by collecting said frame feature data out- 
putted at said DC/partial AC component extracting step. 
[0027] In accordance with a twenty-first aspect of 
is the present invention, there is provided a picture feature 
extraction method comprising: a picture data storing 
step for storing said picture data; and a picture feature 
data storing step using each picture data stored at said 
picture data storing step for storing the still picture fea- 
so ture data generated at said still picture feature extrac- 
tion step and the moving picture feature data generated 
at said moving picture feature extraction step as a pic- 
ture feature data. 

[0028] in accordance with a twenty-second aspect 
25 erf the present invention, there is provided a picture fa** 
ture* extraction* method whereto sakfc resize* pettirs' 
generating step divides 3 still picture* into biocksv and. 
takes out from- each block color data representative of 
each block. 

3d I0029J In accordance with a twenty-third aspect of 
the present invention, there is provided a picture feature 
extraction method wherein in case when said picture 
data is compressed by blocks, said resized picture gen- 
erating step takes out from each block DC components. 
35 [0030] In accordance with a twenty-fourth aspect of 
the present invention, there is provided a pictur^retriev- 
ing method comprising; a stilt picture feature extraction 
step including a resized picture generating step for gen- 
erating a resized picture from still picture data, a fre- 
40 quency analyzing step for executing a frequency 
analysis on said resized picture having been generated 
at said resized picture generating step, and a DC/fcartlal 
AC component extracting step for taWng out DC compo- 
nents and a part of AC components obtained as a result 
45 of the analysis at said frequency analyzing step as a 
picture feature data; a still picture feature data correla- 
tion measuring step for calculating a correlation value 
between the still picture feature data of the stin picture 
within a still picture data base where the still picture fea- 
so ture data of the still picture data having been previously 
generated at said still picture feature extraction step is 
stored together with the still picture data and the still pic- 
ture feature data generated at said still picture feature 
extraction step on the basis of the inputted still picture 
ss data; and a similar still picture retrieving step for retriev- 
ing the still picture within said still picture data base hav- 
ing a high correlation value with respect to the still 
picture feature data generated on the basis of the input- 



5 



BN3DOCID: <EI> .tQ16B88A2J_> 



31-JflN-E006(DI) 10:03 SAMSON & PARTNER 



(FRX)+d9 89 £99465 



5. 025/053 



9 



EP1 018 688 A2 



10 



ted stilt picture data as a candidal* of similar stilt picture. 
10031] In accordance with a twenty-fifth aspect of 
the present invention, there is provided a picture retriev- 
ing method compriang: a moving picture feature extrac- 
tion step including a feature data extracting frame 
selecting step tor taking out some data corresponding to 
the entire frames or a portion of the frames from the 
moving picture data, selecting the data subjected to 
processing, a resized picture generating step for gener- 
ating a resized picture from data corresponding to each 
frame having been taken out, a frequency analyzing 
step for executing a frequency analysis on said resized 
picture having been generated by said resized picture 
generating step, a DC/partial AC component extracting 
step for taWng out DC components and a part of AC 
components obtained as a result of the analysis at said 
frequency analyzing step as a frame feature data, and a 
moving picture feature data producing step for produc- 
ing a moving picture feature data by collecting said 
frame feature data outputted at said DC/partial AC com- 
ponent extracting step; a moving picture feature data 
correlation measuring step for calculating a correlation 
value between the mowing picture feature data of the 
moving picture within a moving picture data base where 
the moving picture feature date of the moving picture 
data having been previously generated at said moving 
picture feature extraction step is stored together withthe 
moving picture data and the moving picture feature data 
generated at said owing picture feature extraction step 
on the basis ol the inputted moving picture data; and a 
similar moving picture retrieving step for retrieving the 
moving picture within said moving picture data base 
having a high correlation value with respect to the mov- 
ing picture feature data generated on the basis of the 
inputted moving picture data as a candidate of similar 
moving picture. . 
100321 In accordance with a twenty-sixth aspect of 
the present invention, there is provided a picture retriev- 
ing method wherein said moving picture feature data 
correlation measuring step has a step of dividing the 
moving picture feature data having been generated by 
said moving picture feature extraction step on thebasls 
of said inputted moving picture data, and a step of cal- 
culating in turn a partial correlation value between a 
part of the moving picture feature-data of the moving 
picture included in said moving picture data base and a 
moving picture feature data of each division of the 
divided moving picture feature data, and said similar 
picture retrieving step eliminates the moving picture 
segment included in said moving picture data base from 
the candidates of similar moving pictures when rt has a 
feature data with a partial correlation value lower than a 
predetermined threshold value. 
[0033] In accordance with a twenty-seventh aspect 
of the present invention, there is provided a picture 
retrieving method wherein said moving picture feature 
data correlation measuring step has a step of dividing 
the moving picture feature data having been generated 



by said moving picture feature extraction step on the 
basis of said inputted moving picture data, and a step of 
calculating in turn a partial correlation value between a 
part of the moving picture feature data of the moving 
s picture included in said moving picture data base and a 
partial moving picture feature data of each division of 
the divided moving picture feature data, and said similar 
picture retrieving step eliminates the moving picture 
segment included in said moving picture data base from 
J0 the candidates of similar moving pictures at the time 
when it ts acKnowledged that a number of partial moving 
picture feature data having a feature data with a partial 
correlation value lower than a predetermined threshold 
value surpasses a predetermined threshold value. 
15 [00341 In accordance with a twenty-eighth aspect of 
the present invention, there is provided a picture retriev- 
ing method comprising: a first picture feature data corre- 
lation measuring step for calculating a correlation value 
between a still picture feature data of a Still picture m 
20 inputted still picture data generated by a still picture fea- 
ture extraction step including a resized picture generat- 
ing Step for generating a resized picture from the 
inputted picture data identified as still picture data, a fre- 
quency analyzing step for executing a frequency analy- 
25 sis on said resized picture having been generated at 
said resized picture generating step, and a DC/partial 
AC component extracting step for taking out DC compo- 
nents and a part of AC components obtained as a result 
of the analysis at said frequency analyzing step as a still 
go picture feature data and a picture feature data of a pic- 
ture within a picture data base where a still picture fea- 
ture data having been generated at said still picture 
feature extraction step on the basis of a still picture data 
is stored together with said still picture data: a similar 
$5 still picture retrieving step for retrieving the picture within 
said picture data base having a high correlation value 
with respect to the picture feature data generated on the 
basis of the inputted picture data asa candidate of sim- 
ilar still picture; a second picture feature data correlation 
40 measuring step for calculating a correlation value 
between a moving picture feature data of a moving pic- 
ture m Inputted moving picture data generated by a 
moving picture feature extraction step including a fea- 
ture data extracting frame selecting step for taking out 
4$ some data corresponding to the entire frames or a por- 
tion of the frames from the picture data identified as 
moving picture data, selecting the data subjected to 
processing, a resided picture generating step for gener- 
ating a resized picture from data corresponding to each 
so frame having been taken out a frequency analyzing 
step for executing a frequency analysis on said resized 
picture having been generated by said resized picture 
generating step, a DC/partial AC component extracting 
step for taking out DC components and a part of AC 
55 components obtained as a result of the analysis at said 
frequency analyzing step as a frame feature data, and a 
moving picture feature data producing step for generat- 
ing a moving pictuw* feature data by collecting said 
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frame feature data outputted at said DC/partial AC com- 
ponent extracting step, and a picture feature data of a 
picture within a picture data base where a moving pic- 
ture feature data having been generated at said moving 
picture feature extraction step on the basis of a moving tf 
picture data is stored together with said mowing picture 
data; and a similar moving picture retrieving step for 
retrieving a moving picture segment within said picture 
data base having a high correlation value with respect 
to the moving picture feature data generated on the 10 
basis of the inputted moving picture data as a candidate 
of similar moving picture. 

[0035] In accordance with a twenty-ninth aspect of 
the present invention, there is provided a picture retriev- 
ing method wherein said second picture feature data is 
Correlation measuring step has a step of dividing the 
moving picture feature data having been generated by 
said moving picture feature extraction step on the basis 
of said inputted moving picture data, and a step of cal- 
culating in turn a partial correlation value between a so 
part of the moving picture feature data of the moving 
picture included in said moving picture data base and a 
moving picture feature data of each division of the 
divided moving picture feature data, and said similar 
picture retrieving step eliminates the moving picture 25* 
segment included in said moving picture data base from 
the candidates of similar moving pictures when it has a 
feature data with a partial correlation value lower than a 
predetermined threshold value. 

[0036] In accordance with a thirtieth aspect of the ao 
present invention* there is provided a picture retrieving 
method wherein said second picture feature data corre- 
lation measuring step has a step of dividing the moving 
picture feature data having been generated by said 
moving picture feature extraction step on the basis of ss 
said inputted moving picture data, and a step of calcu- 
lating in turn a partial correlation value between a part 
of the moving picture feature data of the moving picture 
included in said moving picture data base and a partial 
moving picture feature data of each division of the <o 
divided moving picture feature data, and said similar 
picture retrieving step eliminates the moving picture 
segment included in said moving picture data base from 
the candidates of simitar moving pictures at the time 
when it is acknowledged that a number of partial moving 4s 
picture feature data having a feature data with a partial 
correlation value lower than a predetermined threshold 
value surpasses a predetermined threshold value. 
[0037] in accordance with a thirty-first aspect of the 
present invention* there is provided a picture retrieving so 
method wherein said resized picture generating step 
divides a picture into blocks, and takes out from each 
block color information representative of each block. 
[0038] In accordance with a thirty-second aspect of 
the present invention* there is provided a picture retriev- ss 
ing method wherein said resized picture generating step 
takes out DC components from said each block when 
said picture data is compressed by blocks. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0039] The above and further objects and the novel 
feature of the invention will more fully appear from the 
following detailed description when the same is read in 
connection with the accompanying drawings, in which: 

Fig. 1 is a block diagram showing an example of 
structure of an embodiment of the present invention 
where a picture feature extraction device and a 
method thereof are used; 

Rg. 2 is a block diagram showing an example of 
structure of an embodiment of the present invention 
where a picture feature extraction device and a 
method thereof are used; 

Fig. 3 is a block diagram showing an example of 
structure of an embodiment of the present invention 
where a picture feature extraction device and a 
method thereof are used; 

Fig. 4 is a block diagram showing an example of 
structure of act embodimentof the preset invention 
where. a> pietan^ feattin* extraction* cfeviea ant* ar 
methodrthereot abused; 

Fig* 3T is- a- block- diaoranr sbcwui ng 1 arV of 
stroggqr^ ot^ u a iAxx iim enfof th%pres^iriv€tttfdn 
where a picture feature extraction device and & 
method thereof are used; 

Rg. 6 is a block diagram showing an example- of 
structure of an embodimentof th&present inventor* 
where a picture retrieving device" and x method 
thereof are used; 

Fig. 7 is a block diagram showing art example of 
structure of an embodimentof the-present invention 
where a picture retrieving device and a method 
thereof are used; 

Fig. 8 is a flow chart showing an example of 
processing in the picture retrieving method of the 
present invention; 

Fig. 9 is a block diagram showing an example of 
structure of an embodiment of the present invention 
where a picture retrieving device and a method 
thereof are used; and 

Fig. 10 is a table showing an example of retrieval 
result of the present invention and grid layout based 
method. 

DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

[0040] Referring now to the drawings, a detail 
description of preferred embodiments of the present 
invention will be described in detail. 
[0041] In Fig 1 to Fig, 10. the embodiments of the 
picture feature extraction device, picture retrieving 
device, and methods of generating picture feature data 
and retrieving pictures, respectively, are illustrated, 
[0042] A first embodiment of the picture feature 
extraction device and method thereof in accordance 
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with the present invention will be described in detail wffli 
reference to Fig. 1. 

[0043] The picture feature extraction device in the 
first embodiment as Shawn in Rg. 1 comprises a still 
picture feature extraction section 22 for inputting still 
picture data and generating the feature data of that still 
picture data having been inputted. The still picture fea- 
ture extraction section 22 has a resized picture generat- 
ing means 221 , a frequency analyzing means 222, and 
a DCJpartial AC component extracting means 223. 
[0004] The resized picture generating means 221 
generates a resized picture 12 on the basis of an input 
of still picture data 11. The resized picture at this time 
can taKe any size. As to an example of a method of pro- 
ducing a resized picture, there is a method in which a 
picture is divided into blocks, and only color data repre- 
sentative n' *ach block is taken out In this case, as to 
the repres < ^tive color data for each block, an average 
color, the - frequently used cotor. etc., can be used. 
Furthering: in case when the picture data is com- 
pressed b/ clocks, H is possible to produce a resized 
picture by taking out the DC components from each 
block and it is possible to produce a resized picture 
from said resized picture produced by taking out the DC 
components from each block: Any resolutions of the 
input picture can be allowed by resizing the input picture 
to a fixed size. 

[0046] The frequency analyzing means 222 con* 
ducts a frequency analysis on the resized picture 12 so 
as to calculate frequency components 13. It is also pos- 
sible to divide the resized picture 12 into a plurality of 
areas before executing the frequency analysis. As to a 
method of frequency analysis of the resized picture. 
DOT transformation. Wavelet transformation. Fourier 
transformation, Hadamard transformation, and so forth 
can be used. 

[0046] The DC/partial AC component extracting 
means 223 extracts DC/partial AC components 14 from 
the frequency components 13 having been calculated 
by the frequency analyzing means 222 to generate a 
feature data. As to a specific example of the application 
of the frequency analyzing means 222 and the DC/par- 
tial AC component extracting means 223. the frequency 
analysis can be executed by a two-dimensional OCT 
with respect to the whole size of the resized picture, so 
as to extract the DC components as welt as n AC com- 
ponents from the calculated frequency components. 
Furthermore, it is also possible to extract only the lumi- 
nance coeflidents. or extract only the chrominance 
coefficients, or part of both kinds of coefficients. 
[0047] In accordance with the above-described pic- 
ture feature extraction device and method thereof, the 
frequency analysis is executed on the resized picture, 
and only a part of the acquired coefficients are used, 
whereby picture feature data requiring less memory 
capacity can be made available. 
[0048] A second embodiment of the picture feature 
extraction device and method thereof in accordance 



with the present invention will be described in detail with 
reference to Rg. 2. 

[0049] The picture feature extraction device in the 
second embodiment as shown in Rg. 2 comprises a 
5 moving picture feature extraction section 2 for inputting 
moving picture data and generating the feature data of 
that moving picture data having been inputted. The 
moving picture feature extraction section 2 has a feature 
data extracting frame selecting means 21 , a still picture 
io feature extraction section 22, and a feature data gener- 
ating means 23. As already mentioned, the still picture 
feature extraction section 22 has a resized picture gen- 
erating means 221, a frequency analyzing means 222, 
and a DC/partial AC component extracting means 223. 
i5 Moving picture by definition is made Of a plurality of still 
pictures, and therefore, even when the subject here in 
the second embodiment is moving pictures, it is possi- 
ble to use the still picture feature extraction section 22 in 
this embodiment as in the first embodiment 
zo [0050] The feature data extracting frame selecting 
means 21 as being a constituent of the moving picture 
feature extraction section 2 selects frame data 1 1 sub- 
jected to extraction of feature data on the basis of input- 
ted moving picture data 1 . 
2s [0051] The feature data extracting frame selecting 
means 21 can take out the entire frames or a portion of 
the frames in the moving picture data. The feature data 
extracting frame selecting means can also take out only 
the compressed data corresponding to an interframe 
30 encoded frame (lecture) of which compression proc- 
ess is completed within a single frame in the moving pic- 
ture data. It is also possible to select the all l-Pictur es or 
only a part of the 1-Pictures. Furthermore, as to forward 
prediction encoded frames and bi-directional prediction 
3$ encoded frames of which compression process is not 
completed within a frame, it is possible to produce the I- 
Picture in a pseudo-manner, using the method as intro- 
duced in "On the extraction of DC sequence from 
MPEG Compressed Video" by Boon-Lock Yeo and 
aq Bade Liu (IEEE international Conference on image 
Processing, 1995). In this way, it is possible to produce 
the l-Plcture in a pseudo-manner, and seiect them. 
[0O52J The resized picture generating means 221 
constituting the still picture feature extraction section 22 
45 generates a resized picture on the basis of the still pic- 
ture data 11 selected by the feature data extracting 
frame selecting means 21. The frequency analyzing 
moans 222 decomposes the resized picture 12 into fre- 
quency components 13. The DC/partial AC component 
so extracting means 223 extracts the DC/partial AC com- 
ponents 14 from the frequency components 13 having 
been given by the frequency analyzing means 222. 
[0053] The feature data generating means 23 gen- 
erates feature data 15 by extracting the DC/partial AC 
ss components 14 given by the still picture feature extrac- 
tion section 22 with respect to the whole selected 
frames. 

[0054] In accordance with this embodiment alsa m 
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the picture feature extraction device and method 
thereof, the frequency analysts is executed on the 
resized picture, and only a part of the acquired coeffi- 
cients are used, whereby picture feature data requiring 
less memory capacity can be made available. 
[0055] A thM embodiment of the picture feature 
extraction device and method thereof in accordance 
with the present invention will be described in detail with 
reference to Fig. 3. 

10056] The picture feature extraction device in the 
third embodiment as shown in Rg. 3 comprises a still 
picture feature extraction section 22 for inputting still 
picture data and generating the feature data of that still 
picture data having been Inputted, and a feature data 
added still picture data base storing section 6- The fea- 
ture data added still picture data base storing section 6 
has a still picture data storing section 51 fp-r storing the 
inputted still picture data, and a still picture feature data 
Storing section 62 for storing the still picture feature data 
having been generated at the still picture feature extrac- 
tion section 22: The still picture feature extraction sec- 
tion 22 has a resized picture generating means 221. a 
frequency analyzing means 222, and a DOj^artial AC 
component extracting means 223. 
[0057] In the still picture feature extraction section 
22, the resized picture generating means 221 generates 
a resized picture on the basis of the mputiecistill picture 
data, the frequency analysing means 223 conducts a 
frequency analysis of the resided picture- having been 
generated by the resized picture generating means 221 
and the DC/partial AC component extracting means 223 
takes out the DC components and a part of AC compo- 
nents obtained as a result of frequency analysis. Thus 
the still picture feature extraction section 22 generates 
the still picture feature data. 

[0058] The still picture feature data having been 
generated is to be stored in the still picture feature data 
storing section 62. while the still picture data used in 
generating the still picture feature data is to be stored in 
the still picture data storing section 61. 
[0059] In accordance with this embodiment also, in 
the picture feature extraction device and method 
thereof, the frequency analysis is executed on the 
resized picture, and only a part of the acquired coeffi- 
cients are used, whereby picture feature data requiring 
less memory capacity can be made'availabie. 
[0060] As shown in Fig. 4. the picture feature 
extraction device in a fourth embodiment of the picture 
feature extraction device and method thereof, has a 
moving picture feature extraction section 2, and a fea- 
ture data added moving picture data base storing sec- 
tion 3. As to the moving picture feature extraction 
section 2, the description thereof will be omitted. 
[0061] In this embodiment, feature-data 1 5 is gener- 
ated from moving picture data 1. and both the feature 
data 15 and the moving picturedata 1 are stored in the 
feature data added moving picture data base storing 
section 3. 



[00621 The feature data added moving picture data 
base storing section 3 is composed of a moving picture 
data stonng section 31. and a moving picture feature 
data storing section 32. With respect to various moving 
5 p.cture data 1 . the feature data added moving picture 
data base storing section 3 stores both the moving pic- 
ture data 1 and the feature data 15 having been gener- 
ated by the feature extraction section 2 on the basis of 
the moving picture data 1 . 
io [0O63J The moving picture data storing section 31 
which is one of the two constituents of the feature data 
added moving picture data base storing section 3 stores 
the moving picture data 1. The moving picture feature 
data storing section 32 stores the feature data 15 hav- 
w ing been generated by the feature extraction section 2 
on the basis of the moving picture data 1 . 
[0064] In accordance with the picture feature 
extraction device comprising the above mentioned com- 
ponents, data corresponding to the entire frames or a 
20 P 0 *' 0 " <* foe **"ies is taken out from the moving pic- 
ture data by the feature data extracting frame selecting 
means 21. Then the resized picture generating means 
221 generates a resized picture from the data corre- 
sponding to each frame being taken out Then the fre- 
25 quency analyzing means 222 executes the frequency 
analysis of the resized picture having been produced by 
the resized picture generating means 2£t. Then the 
DOjpartial AC component extracting means 223 takes 
out the DC components and a part of AC components 
90 obtained as a result of the frequency analysis, and the 
feature data generating means 23 is to execute the 
process with respect to the whole selected frames in 
order to generate the feature data 15. 
[0065] In the picture feature extraction device of this 
as embodiment the frequency analysis is executed on the 
resteed picture of part of the frames, and only a part of 
the acquired coefficients are used, whereby picture fea- 
ture data requiring less memory capacity can be made 
available. 

40 [opeej A fifth embodiment of the picture feature 
extraction device and method thereof in accordance 
with the present invention wilt be described in detail with 
reference to Fig. 5. 

[0067] As illustrated in Rg. 5. the picture feature 
**s extraction device of the fifth embodiment comprises a 
picture feature extraction section 10 and a feature data 
added picture data base storing section S. The picture 
feature extraction section 10 comprises a picture data 
identifying section 100. a still picture feature extraction 
so section 22, and a moving picture feature extraction sec- 
tion 2 The picture data in this embodiment isdefined as 
including the moving picture data and the still picture 
data, and the moving picture data would be composed 
of a plurality of still pictures. 
& P068] lntoiserrtoodimem.asmep^^ 

trying section 100 receives tha picture data 7, it deter- 
mines whether the inputted picture data is of still 
pictures, or rrxaring pkaura* In casa when thev pidum 
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data is of still pictures, the picture data is inputted to the 
still picture feature extraction section 22, while in case 
when the picture data is of mewing pictures, the picture 
data is inputted to the moving picture feature extraction 
section 2. 

[0069] trt the still picture feature extraction section 
22 the resized picture generating means 221 generates 
a resized picture on the basis of the inputted still picture 
data/ the frequency analyzing means 222 conducts a 
frequency analysis on the resized picture having been 
generated by the resized picture generating means 221 ( 
and the DC/partial AC component extracting means 223 
takes out the DC components and a part of AC compo- 
nents obtained as a result ol the frequency analysis, 
thus generating the feature data 
roOTO] In the moving picture feature extraction sec- 
tion 2, data corresponding to the entire frames or a por- 
tion of the frames is taken out from the moving picture 
data by the feature data extracting frame selecting 
means 21. Then the resized picture generating means 
221 'generates a resized picture from the data corre- 
sponding to each frame being taken out Then the fre- 
quency analyzing means 222 executes the frequency 
analysis on the resized picture having been produced 
by the resized picture generating means 221 . Then the 
DC/partial AC component extracting means 223 takes 
out the DC components and a part of AC components 
obtained as a result of the frequency analysis, and the 
feature data generating means 23 is to execute the 
process with respect to the entire frames being selected 
In order to generated the feature data. 
[0071] In accordance with the above-mentioned 
process, the feature data from the still picture feature 
extraction section and the feature data 15 from the mov- 
ing picture feature extraction section 2 are each stored 
in a picture feature data storing section 52 of the feature 
data added picture data base storing section 5. Further- 
more, the picture data composed of moving and still pic- 
tures, respectively are stored In a picture data storing 
section 51 . , . 

[0072] Thus* when the still picture Is inputted, the 
frequency analysis Is executed on the resized picture* 
and only a part of the acquired coefficients are used in 
generating the feature data, while when the moving pic- 
ture is inputted, the frequency analysis is executed on 
the resized picture of a' part of the frames, and only a 
part of the acquired coefficients are used in generating 
the feature data. Therefore, the still picture feature data 
as well as moving picture feature data requiring less 
memory capacity can be made available. 
[0073] Referring now to fig. 6. a first embodiment 
of the picture retrieving device and method thereof will 
be described- T ' 

[0074] As shown in Fig. 6. in this embodiment the 
picture retrieving device comprises a still picture feature 
extraction section 22 for inputting still picture data and 
generating the feature data of that still picture data hav- 
ing been inputted, a feature data added still picture data 



base storing section 6 including a still picture data stor- 
ing section 61 for storing the inputted still picture data 
and a still picture feature data storing section 62 for stor- 
ing the still picture feature data having been generated 
5 from the still picture feature extraction section 22, and a 
still picture correlation measuring section 41 for calcu- 
lating a correlation value between the stfll picture fea- 
ture data having been generated from the inputted still 
picture data by the still picture feature extraction section 
10 22 and the still picture feature data stored in the feature 
data added still picture data base storing section 6. 
[0075] tn the still picture feature extraction section 
22 the resized picture generating means 221 generates 
a resized picture on the basis of the inputted still picture 
is data, the frequency analyzing means 222 conducts a 
frequency analysis on the resized picture having been 
generated by the resized picture generating means 221. 
and the DC/partial AC component extracting means 223 
takes out the DC components and a part of AC compo- 
se nents obtained as a result of the frequency analysis. 
Thus the still picture feature extraction section 22 gener- 
ates the still picture feature data with respect to the 
inputted still picture data 11. 

[0076] In the feature data added still picture data 
25 base storing section 6. through the same process as 
described in the above third embodiment of the picture 
feature extraction device, the still picture has been 
stored in the still picture data storing section 61 , and the 
still picture feature data calculated by the still picture 
so feature extraction section 22 using the still picture 
stored in the still picture data storing section 61 has 
been stored in the still picture feature data storing sec- 
tion 62. 

10077] The still picture correlation measuring sec- 
36 tion 41 is to calculate the correlation value between the 
feature data having been generated from the inputted 
still picture data by the still picture feature ^traction 
section 22 and the feature data having been generated 
from the picture Included in the picture data base. Then 
40 the still picture correlation measuring section 41 is to 
retrieve the still picture in the data base having a high 
correlation value as a candidate of a similar still picture. 
loom Referring now to Fig. 7 and Fig, 8, a second 
embodiment of the picture retrieving device and method 
45 thereof will be described. Fig. 7 is a block diagram 
showing the structure of the picture retrieving device in 
the second embodiment of the picture retrieving device 
and method thereof. Fig. S is a flow chart showmg a 
process of correlation measurement in the picture 
so retrieving method. t , . 

[00791 As shown in Fig. 7. the picture retrieving 
device in the second embodiment comprises a feature 
extraction section 2, a feature data added moving pic- 
ture data base storing section 3. and a correlation 
55 measur ing section 4 for calculating a correlation value 
between feature data 1 7 in the moving picture data base 
and feature data 15 having been generated by the fea- 
ture extraction section 2 on the basis of the Inputted 
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moving picture data 1 . 

[0080] First, the moving picture data to be used far 
a retrieval is inputted to the feature extraction section 2, 
from which the feature data 15 is generated. In execut- 
ing the retrieval, feature data is generated form the ref- 
erence moving picture, so as to be inputted to the 
correlation measuring section 4. 
[0081] The correlation measuring section 4 calcu- 
lates a correlation value between the feature data 17 in 
the moving picture data base and the feature data 15 
having been generated by the feature extraction section 
2 on the basis of the inputted moving picture data T 
Then storage location data 18 of a partial moving pic- 
ture being retrieved is outputted as a retrieving result 
[0082] The operation of the moving picture correla- 
tion measuring section 4 will be descrfeed in detail with 
reference to Fig. a 

[0083] In accordance with the picture retrieving 
method of the present invention, the processing proc- 
ess for obtaining the correlation value between the fea- 
ture data having been generated from the inputted 
moving picture data and a part of the feature data hav- 
ing been generated from the moving picture included in 
the moving picture data base consists of three steps in 
which the feature data having been generated from the 
inputted moving picture is divided in a first step, partial 
correlation is calculated in sequence for each of the 
divided parts of the feature data in a second step, and 
the moving picture in the data base having the feature 
data corresponding to the partial correlation lower than 
the predetermined threshold value is eliminated from 
the candidate pictures Jn a third step. By adapting such 
method, the candidate pictures as a result of a retrieval 
can be narrowed down in number through a less 
number of matching process, as compared to the case 
in which the matching process is executed on the entire 
designated moving pidure segments in a round robin 
manner. This contributes to shortening the processing 
time required in retrieving the moving pictures. The 
details of this processing process wilt be described in 
the following. 

10084] First, in me Initial setting, identification (ID) 
numbers for bringing out the moving picture feature data 
Subjected to retrieval is initialized (step Si). In this case, 
the ID number is a unique number given individually to 
each moving picture data in generating the moving pic- 
ture data basa The feature data 15 should also be pre- 
viously acquired. 

[0085] Next, the moving picture feature data sub- 
jected to retrieval, which corresponds to the ID is 
acquired, and a pointer indicating the acquisition loca- 
tion of the partial feature data subjected to retrieval (to 
be referred to as target partial feature data) is arranged 
in the front (step S2)_ 

[0086] The feature data of a predetermined time 
length is acquired from the reference partial feature 
data in the front This data is to be referred to as refer- 
ence partial feature data. Furthermore, the target partial 



feature data is acquired from the point indicated by the 
pointer showing the acquisition location (step S3). 
[0087] Then the correlation between the reference 
partial feature data and the target partial feature data 
* acquired in step S3 will be determined (step S4). 

[0088] When the acquired correlation value is lower 
than the predetermined threshold value, (the condition 
will be referred to as a mismatch judgement), the pres- 
ence of the target partial feature data to be acquired 
10 next will be checked in step S5. When the presence of 
the next target feature data is acknowledged, the acqui- 
sition location pointer of the target partial feature data is 
shifted by a predetermined value (step S6), before 
returning to step S3. On the other hand, when the next 
is target partial feature data is not there, another ID is 
added, and the same process is conducted on the fol- 
lowing moving picture feature data (step S13). However, 
in case when the matching process is conducted on the 
entire moving picture feature data within the data base, 
so the process is to be terminated (step S12). 

[0080] In case when the correlation value is deter- 
mined as larger than the predetermined threshold 
value, the next target partial feature data and the refer- 
ence partial feature data following the acquisition loca- 
ls tion followed by a certain time length is acquired (step 
S7), and the correlation will be determined (step S8). As 
to the subsequent events, the matching process, ia 
repeated in the* same manner until encountering a mis- 
match, and when the entire reference partial feature 
so data match (step S9), the result will be outputted as a 
retrieving result (Step S10). 

[0090] In this occasion, it is possible to output the 
moving picture data ID number including the retrieving 
result the start location, the end location, etc. in the 

35 moving picture data, correlation value, eta 

10091] In case when the mismatch judgement is 
given, it is to be determined whether a predetermined 
number of mismatch judgements or a predetermined 
period of time having been separately set has sur- 

40 passed (step S11). if not surpassed, the process 
returns to step S7. If surpassed, the process returns to 
step S5. In step Si 1 , temporal mismatches caused by a 
difference in a temporal position of a frame, a difference 
in camera work, a change in the feature data due to 

45 screen noises, etc. can be resolved by appropriately 
setting the number of times and a period of time with 
respect to the mismatch judgements, Furthermore, set- 
ting the number of times and a period of time to "0" Is 
the same as eliminating step Si 1, and thus H is alsa 

so possible to simplify the circuit design. 

[0092] On the other hand, it is also possible to con- 
sider the time length in acquiring the reference partial 
feature data as of the entire reference feature data, so 
as to compare the correlation with the retrieving partial 

55 feature data one a* a time-, thus executing the retrieval: 
[0093] A third embodiment of the picture retrieving 
device and method thereof in accordance with the 
present invention will ba described; detail Aatifr refer. 
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ence to Rg. 9. 

[0094] As shown in Fig. 9, the picture retrieving 
device erf this embodiment comprises a picture feature 
extraction section 10, a feature data added picture data 
base storing section S including a picture data storing 
section 51 tor storing still pictures and moving pictures 
and a picture feature data storing section 52 storing the 
still picture leature data and the moving picture leature 
data having been previously generated by the picture 
feature extraction section 10 on the basis of the picture 
data stored in the picture data storing section, a moving 
picture correlation measuring section 4 for calculating a 
correlation value between feature data 17 in the moving 
picture data base and feature date 1 5 having been gen- 
erated by the feature extraction section 2 with respect to 
the inputted moving picture data 1, and a still picture 
correlation measuring section 41 for calculating the cor- 
relation value between the feature data 1 7 In the picture 
data base and the feature data having been generated 
by the picture feature extraction section 10 on the basis 
of the inputted still picture data 1 1 . 
[0005] First picture data 1 used in a retrieval is 
inputted to generate the feature data 15 by the leature 
extraction section 2. In case when the picture data for 
reference is of moving pictures, feature data is gener- 
ated from the moving picture by the feature extraction 
section 10 to be inputted to the moving picture feature 
data correlation measuring section 4. In case when the 
picture data for reference is of still pictures, feature data 
is generated from the still picture by the feature extrac- 
tion section 10 to be inputted to the still picture feature 
data correlation measuring section 41 . 
[0096] 7116 ™ovmg picture correlation measuring 
section 4 is to calculate the correlation value between 
the feature data 17 within the moving picture data base 
and the feature data 1 5 generated by the feature extrac- 
tion section 2 on the basis of the i nputted moving picture 
data 1. Then it outputs storage location data 18 on the 
partial moving picture having been retrieved as a 
retrieving result The explanation on the processing 
operation in this embodiment will be omitted tor the 
explanation has already been given In the description of 
the second embodiment of the moving picture feature 
extraction device. 

[0097] The still picture correlation measuring sec- 
tion 41 Is to calculate the correlation value between the 
feature data having been generated from the inputted 
still picture data by the feature extraction section 1 0 and 
the feature data generated by the picture included in the 
picture data base, so as to retrieve a still picture within 
the data base having a high correlation value as a can- 
didate for a similar still picture. 
[0098] In the following, an example of an embodi- 
ment of the present invention will be described. 
[0099] An experiment is conducted using 50 que- 
ries among 5466 still pictures, where the correct results 
for those 50 queries are pre-defined by human judge- 
ment based on similarity. 



pi 00] First all still pictures are divided into 64 (8 x 
8) blocks and average color of each block is taken out. 
The derived average colors are transformed into a 
series of coefficients by 8 x 8 OCT. Three DC coeffi- 
5 dents, five luminance AC coeffidems and two compo- 
nents per each chrominance AC coefficients normalized 
within the range of [0. 255] for each component are 
used as feature data. 

[0101] Figure 10 shows a comparison between 
i o using DOT coefficients on a picture resized to 8 x 8 pix- 
els and 2 x 2, 3 * 3, 4 x 4 grid layout and average color 
within the range of [0. 255] for each color component as 
feature data. Correct retrieval rate means how many of 
the correct pictures are detected within top K ranks 
is when all the images are sorted according to a weighted 
Euclidean distance between feature data of a query pic- 
ture and stored pictures. Here K denotes a number of 
correct results multiplied by four. Figure 10 indicates 
that DOT coefficients perform better than any grid 
so . based feature data with smaller size of feature data. 
P1021 in the following, another example of an 
embodiment of the present invention will be described. 
[0103] An experiment is conducted using 20 hours 
worth of video data being encoded by MPEG-i . MPEG- 
25 2 encodings, where a certain video segment corre- 
sponding to a particular article, TV commercial, and an 
opening of a particular program is designated in retriev- 
ing a video segment seemingly identical to the desig- 
nated video segm ent 
30 [0104] First a time code and DC components are 
extracted from each compressed data corresponding to 
I- Picture within the moving picture as a whole subjected 
to retrieval. In this occasion, the resized picture having 
been generated by extracting the DC components is fiir- 
$s ther resized to a fixed ske of 8 pixel * 8 pixel and 
exposed to DOT to obtain three DC luminance and 
chrominance components [to be referred to as DC_Y 
DC_Cb, and DC_Cr), as well as 5 luminance AC 
components. 2 components per each chrominance AC 
do component (to be referred to as AC_Y. AC__Cb. and 
AC_Cr) ( which are normalized within the range of [0, 
2551 for each DC and AC component and stored 
together with the time codes, respectively. For each l- 
Picture. only a total of 1 6 bytes including 1 2 bytes of foa- 
45 turedada and 4 bytes of time code are required. There- 
fore, when the 1-Pictures exist in the intervals of 0.5 
seconds, the size of data of the feature data corre- 
sponding to the amount of video worthy of a portion for 
a single day can be managed within about 2.8 mega 
so bytes. 

PJ1 05] The feature data of the head frame of the ref- 
erence moving picture goes through a matching proc- 
ess with the feature data of the retrieving target moving 
picture in a unit of I-Picture. A candidate for the aiming 
55 moving picture is defined by determining the ones of 
which difference value of DC_Y. 0C_Cb. and DC Cr 
(to be referred to as DC difference value), and the differ- 
ence value of AC_Y t AC_Cb andAC_Cr meet a cer- 
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tain specific condition {threshold value). 
[0106J When a certain frame is determined as a 
candidate for the aiming moving picture the frames 
ahead are to be inspected in time series, in case when 
the frame is not determined as a candidate for the aim- s 
ing moving picture, the same process as described is to 
be repeated on the next frame. After that, the ones of 
which DC difference values meet the specified condition 
are to bo regarded as the matched ones, and the 
inspection is to be conducted in due order. For the pur- iq 
pose of reducing the number of times conducting the 
matching process, AC_Y, AC Cb, and AC_Cr can be 
disregarded. 

[0107] Considering the errors due to sampling 
points, when the mismatch is determined three frames is 
in a row. the frames are eliminated from the candidates. 
When the difference of feature data with respect to a 
length corresponding to the designated moving picture 
segment meets the specified condition (threshold 
value), it ie to be determined as the aiming moving pic- so 
ture segment in response to which the time codes for 
the start point and the end point of that certain length 
are outputted as a retrieving result Furthermore, in 
case when the aiming moving picture segment is 
retrieved, the start point for the next searching can be 2S 
set as the frame next to the end point of the? detected 
section. 

[01081 According to such process, camparetttathe' 
case in which the matching process is conducted in a 
round robin manner over the entire designated moving 30 
picture segment it Is possible to reduce the amount of 
processing by a considerable extent 
[0109] As an experiment was conducted on 11 
pairs of moving picture segments which seem identical, 
it was proved with the whole 1 1 pairs that when one of ss 
the two moving picture segments in a single pair is des- 
ignated, it was possible to obtain the other one in the 
pair. Furthermore even when the length of the aiming 
moving picture segment was set as over 3 seconds, no 
excessive retrieving was noted. 40 
[Oil 0J The above described embodiments are 
merely an example of the preferred embodiment of the 
present invention. The present invention, therefore, is 
not limited to such illustrative description but changes 
and variations may be made without departing from the « 
subject matter of the invention, in any of the above 
described embodiments, the application of the present 
invention can be extended to a retrieving of similar mov- 
ing picture segments by defining the distance between 
the feature data instead of excluding the ones not meet- so 
ing the condition. 

[0111] As it has been made dear in the above 
description, in accordance with the picture feature 
extraction device and method thereof of the present 
invention, the feature data is generated through gener- 55 
ating a resized picture from the picture data, executing a 
frequency analysis on the generated resized picture, 
and taking out DC components andapart of AC compo- 



nents obtained as a result of the analysis. 
E01 12] Furthermore, in accordance with the picture 
feature extraction device and method thereof, data cor- 
responding to the entire frames or part of the frames in 
the picture data is taken out, and the resized picture is 
generated from the data corresponding to each frame 
being taken out The frequency analysis is conducted 
on the generated resized picture, the DC components 
and a part of AC components obtained as a resuit of the 
frequency analysis are taken out and this process is 
implemented on the whole frames being selected. 
[0113J in accordance with the picture retrieving 
device and method thereof of the present invention, It is 
possible to eliminate the pictures from the candidates 
when it becomes obvious that the picture would have no 
chance of being identified as a similar picture in the 
process of calculating the correlation. According to such 
process, compared to the case in which the matching 
process is conducted in a round robin manner over the 
entire designated moving picture segment, it is possible 
to reduce the number of matching process for narrowing 
down the retrieving candidates to be regarded as the 
retrieving result thus reducing the amount of procfcssr 
ing by a considerable extent 

[01 1*] Fl^Brnftore; inacc^^cd^vfth^tn^pn-e^ 
.irrverrtiort. the frequency analysis is; executed orr the 
resteed picture, and only a part of the acqinna* coeffi- 
cients are used/whereby picture feature data requiring, 
less memory capacity can be made available. 
[01 1 5] The present invention can widely apply to a 
variety of purposes such as an investigation on a broad- 
casting frequency, obtaining a statistics on some broad- 
casting information, a tendency analysis, etc, for a 
particular video image such as TV commercial pictures, 
news articles, opening of particular programs, etc. 
[0116] Furthermore, the present invention can 
apply to a retrieving of similar picture etc., for retrieving 
a similar color layout picture for reference within the 
stored pictures. 

[0117] While the preferred embodiments of the 
invention have been described using specific terms, 
such description is far illustrative purposes only, and it is 
to be understood that changes and variations may be 
made without departing from the spirit or the scope of 
the following daims. 

Claims 

1 - A picture feature extraction device comprising: 

a resized picture generating means generating 
a resized picture from still picture data: 
a frequency analyzing means executing a fre- 
quency analysis on said resized picture having 
been generated by said resized picture gener- 
ating means; and 

a DC/partial AC component extracting means 
taking out DC components and a part of AC 
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components obtained as a result of the analy- 
sis by said frequency analyzing means as a 
picture feature data. 

2. A picture feature extraction device as claimed in 
claim 1 4 comprising: 

a still picture data base including a still picture 
data storing means storing said still picture 
data, and a still picture feature data storing 
means storing said still picture feature data 
generated lor each still picture data stored in 
said still picture data storing means. 

3. A picture feature extraction device comprising: 

a feature data extracting frame selecting 
means taking out some data corresponding to 
the entire frames or a portion of the frames 
from the moving picture data, selecting the 
data subjected to processing; 
a resized picture generating means generating 
a resized picture from data corresponding to 
each frame having been taken out; 
a frequency analyzing means executing a fre- 
quency analysis on said resized picture having 
been generated by said resized picture gener- 
ating means; 

a DC/partial AC component extracting means 
taking out DC components and a part of AC 
" components obtained as a result of the analy- 
sis by said frequency analyzing means as a 
frame feature data; and 
a moving picture feature data producing means 
producing a moving picture feature data by col- 
lecting said frame feature data outputted from 
said DC/partial AC component extracting 
means. 
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4. A picture feature extraction device as claimed in 
claim 3, comprising: 

a moving picture data base including a moving 
picture data storing means storing said moving 
picture data, and a moving picture feature data 
storing means storing said moving picture fea- 
ture data generated for each moving picture 
data stored in said moving picture data storing 
means. 

5* A picture feature extraction device comprising: 

a picture data identifying means identifying 
whether the inputted picture data is of still pic- 
tures or of moving pictures; 
a still picture feature extraction means includ- 
ing a resized picture generating means gener- 
ating a resized picture from the picture data 
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identified as still picture data by said picture 
data identifying means, a frequency analyzing 
means executing a frequency analysis on said 
resized picture having been generated by said 
rested picture generating means, and a 
DC/partial AC component extracting means 
taking out DC components and a part ol AC 
components obtained as a result of the analy- 
sis by said frequency analyzing means as a still 
picture feature data; and 
a moving picture feature extraction means 
including a feature data extracting frame 
selecting means taking out some data corre- 
sponding to the entire frames or a portion of the 
frames from the picture data identified as mov- 
ing picture data by said picture data identifying 
means, selecting the data subjected to 
processing, a resized picture generating 
means generating a resized picture from data 
corresponding to each frame having been 
taken out a frequency analyzing means exe- 
cuting a frequency analysis on said resized pic- 
ture having been generated, a DC^artial AC 
component extracting means taking out DC 
components and a part of AC components 
obtained as a result of the analysis by said fre- 
quency analyzing means as a frame feature 
data, and a moving picture feature data pro- 
ducing means generating a moving picture fea- 
ture data by collecting said frame feature data 
outputted from said DCfcartial AC component 
extracting means. 

A picture feature extraction device as claimed in 
claim 5, comprising: 

a picture data base including a picture data 
storing means storing said picture data* and a 
picture feature data storing means using each 
picture data stored in said picture data storing 
means for storing the stilt picture feature data 
generated by said still picture feature extraction 
means and the moving picture feature data 
generated by said moving picture feature 
extraction means as a picture feature data, 

A picture feature extraction device as claimed in 
claim 2, wherein 

said resized picture generating means 
divides a still picture into blocks, and takes out color 
data representative of each block from each block. 

A picture feature extraction device as claimed in 
claim 4 or 6, wherein 

said resized picture generating means 
divides a still picture into blocks, and takes out color 
data representative of each block from each block. 
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A picture feature extraction device as claimed in 
claim 2, 4. or 6 wherein 

in case when said picture data is com- 
pressed by blocks, said resized picture generating 
means takes out DC components from each block: 



10. A picture retrieving device comprising: 

a still picture generating means including a 
resized picture generating means generating a 
resized picture from still picture data, a fre- 
quency analyzing means executing 'a fre- 
quency analysis on said resized picture having 
been generated by said resized picture gener- 
ating means, and a DGfcartial AC component 
extracting means taking out DC components 
and a part of AC components obtained as a 
result of the analysis by said frequency analyz- 
ing means as a picture feature data; 
a still picture data base including a still picture 
feature data storing means storing the still pic- 
ture feature data previously being generated by 
said still picture feature extraction means on 
the basis of the still picture data, and a still pic- 
ture data storing means storing said still picture 
data; and 

a still picture feature dam correlation measur- 
ing means calculating a correlation value 
between the still picture feature data generated 
by said still picture feature extraction means on 
the basis of the inputted still picture data and 
the still picture feature data stored in said still 
picture data base, the still picture feature data 
correlation measuring means retrieving the still 
picture within the still picture data base having 
a high correlation value as a candidate of simi- 
lar still picture. 
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11- A picture retrieving device comprising: 

a moving picture feature extraction means 
including a feature data extracting frame 
selecting means taking out some data corre- 
sponding to the entire frames or a portion of the 
frames from the moving picture data, selecting 
the data subjected to processing, a resized pic- 
ture generating moans generating a resized 
picture from data corresponding to each frame 
having been taken out. a frequency analyzing 
means executing a frequency analysis on said 
resized picture having been generated by said 
resized picture generating means, a DC/partial 
AC component extracting means taking out DC 
components and a part of AC components 
obtained as a result of the analysis by said fre- 
quency analyzing means as a frame feature 
data, and a moving picture feature data pro- 
ducing means producing a moving picture fea- 
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ture data by collecting said frame feature data 
outputted from said DCfcartial AC component 
extracting means; 

a moving picture data base including a moving 
picture feature data storing means storing the 
moving picture feature data previously being 
generated by said moving picture feature 
extraction means on the basis of the moving 
picture data, and a moving picture data storing 
means storing said moving picture data; and 
a moving picture feature data correlation meas- 
uring means calculating a correlation value 
between the moving picture feature data gener- 
ated by said moving picture feature extraction 
means on the basis of the inputted moving pic- 
ture data and the moving picture feature data 
stored in said moving picture data basa the 
moving picture feature data correlation meas- 
uring means retrieving the moving picture seg- 
ment within the moving picture data base 
having a high correlation value as a candidate 
of similar moving picture segment 

12. A picturfrretrieving dewca comprising: 

a picture data identifying, mean* ic^rttifyino; 
whether the inputted^ picture data is of still pic- 
tures or of moving pictures; 
a still picture feature extraction means includ- 
ing a resized picture generating means gener- 
ating a resized picture from the picture data 
identified as still picture data by said picture 
data Identifying means, a? frequency analyzing 
means executing a frecpency analysis on said 
resized picture having been generated by said 
resized picture generating means, and a 
DCVpartial AC component extracting means 
taking out DC components and a part of AC 
components obtained as a result of the anaty* 
sis by said frequency analyzing means as a still 
picture feature data; 

a moving picture feature extraction means 
including a feature data extracting frame 
selecting means taking out some data corre- 
sponding to the entire frames or a portion of the 
frames from the picture data identified as mov- 
ing picture data by said picture data identifying 
means, selecting the data subjected to 
processing, a resized picture generating 
means generating a resized picture from data 
corresponding to . each frame having been 
taken out, a frequency analyzing means exe- 
cuting a frequency analysis on said resized pic- 
ture having been generated by said resized 
picture generating means, a DGfcirtial AC 
component extracting means taking out DC 
components and a part of AC components 
obtained a& a result ct the analysis, by said frer: 
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quency analyzing means as a frame feature 
data, and a moving picture feature data pro- 
ducing means generating a moving picture fea- 
ture data by collecting said frame feature data 
outputted from said DCftartiaJ AC component 5 
extracting means; 

a picture data base including a picture feature 
data storing means storing the still picture fea- 
ture data previously being generated by said 
still picture feature extraction means on the 10 
basis of the still picture data and the moving 
picture feature data previously being generated 
by said moving picture feature extraction 
means on the basis of the moving picture data, 
and a picture data storing means storing said is 
still picture data and said moving picture data; 
a still picture feature data correlation measur- 
ing means calculating a correlation value 
between the still picture feature data generated 
by said still picture feature extraction means on so 
the basis of the inputted still picture data and 
the picture feature data stored in said picture 
data base; and 

a moving picture feature data correlation meas- 
uring means calculating a correlation value s$ 
between the moving picture feature data gener- 
ated by said moving picture feature extraction 
means on the basis ot the inputted moving pic- 
ture data and the picture feature data stored in 
said picture data base, w 
said still picture feature data correlation meas- 
uring means retrieving the still picture within 
the picture data base having a high correlation 
value as a candidate of similar still picture, 
said moving picture feature data correlation 3s 
measuring means retrieving the moving picture 
segment within the picture data base having a 
high correlation value as a candidate of similar 
moving picture segment 

40 

13, A picture retrieving device as daimed In claim 1 1 or 
12, wherein 

said moving picture feature data correlation 
measuring means divides the moving picture tea-, 
ture data having been generated by said moving <ts 
picture feature extraction means on the basis of 
said inputted moving picture data, and calculates in 
turn a partial correlation value between a part of the 
moving picture feature data of the moving picture 
included in said moving picture data base and a so 
moving picture feature data of each division of the 
divided moving picture feature data, upon which 
eliminating the moving picture segment included in 
said moving picture data base from the candidates 
of similar moving pictures when it has a feature data 55 
with a partial correlation value lower than a prede- 
termined threshold value. 
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14. A picture retrieving device as claimed in daim 1 T or 
12, wherein 

said moving picture feature data correlation 
measuring means divides the moving picture fea- 
ture data having been generated by said moving 
picture feature extraction means In respect with 
said inputted moving picture data, and calculates in 
turn a partial correlation value between a part of the 
moving picture feature data of the moving picture 
included In said moving picture data base and a 
partial moving picture feature data of each division 
of the divided moving picture feature data, upon 
which eliminating the moving picture segment 
induded in said moving picture data base from the 
candidates of similar moving pictures at the time 
when it is acknowledged that a number ol partial 
moving picture feature data having a feature data 
with a partial correlation value lower than a prede- 
termined threshold value surpasses a predeter- 
mined threshold value* 

15, A picture retrieving device as claimed in one ot 
daims 10 to 14, wherein 

said resized picture generating means 
divides a picture into blocks, and takes out from 
each block color information representative of each 
block. 

16, A picture retrieving device as daimed in one of 
claims 10 to 14, wherein 

said resized picture generating means takes 
out DC components from said each block when 
said picture data is compressed by blocks. 

17, A picture feature extraction method comprising: 

a resized picture generating step lor generating 
a resized picture from still picture data; 
a frequency analyzing step for executing a fre- 
quency analysis on said resized picture having 
been generated by said resized picture gener- 
ating step; and 

a DC/partial AC component extracting step for 
taking out DC components and a part of AC 
components obtained as a result of the analy- 
sis at said frequency analyzing step as a pic- 
ture feature data. 

18. A picture feature extraction method as claimed in 
daim 17, comprising: 

a still picture data storing step for storing said 
still picture data; and 

a still picture feature data storing step for stor- 
ing said still picture feature data generated for 
each still picture data stored at said still picture 
data storing step. 
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19. A picture feature extraction method comprising: 

a feature data extracting frame selecting step 
tor taking out some data corresponding to the 
entire frames or a portion of the frames from s 
the moving picture data, selecting the data sub- 
jected to processing; 

a resized picture generating step for generating 
a rested picture from data corresponding to 
each frame having been taken out; w 
a frequency analyzing step for executing a fre- 
quency analysis on said resized picture haying 
been generated at said resized picture gener- 
ating step; 

a DC/partial AC component extracting step for ts 
taking out DC components and a part of AC 
components obtained as a result of the analy- 
sis at said frequency analyzing step as a frame 
feature data; and 

a moving picture foature data producing step so 
for producing a moving picture feature data by 
collecting said frame feature data oirtputted 
from said DC/partial AC component extracting 
step. 



ture data at said picture data identifying step, 
selecting the data subjected to processing, a 
resized picture generating step for generating a 
resized picture from data corresponding to 
each frame having been taken out, a frequency 
analyzing step for executing a frequency analy- 
sis on said resized picture having been gener- 
ated at said resized picture generating step, a 
DC/partial AC component extracting step for 
taking out DC components and a part of AC 
components obtained as a result of the analy- 
sis at said frequency analyzing step as a frame 
feature data, and a moving picture feature data 
producing step for generating a moving picture 
feature data by collecting said frame feature 
data outputted at said DCfcartjeJ AC compo- 
nent extracting step. 

22- A picture feature extraction method as claimed in 
claim 21 , comprising; 



20- A picture feature extraction method as claimed irr 
claim 19, comprising: 

.a moving picture data storing step for storing 
said moving picture data; and 
a moving picture feature data storing step for 
storing said moving picture feature data gener- 
ated for each moving picture data stored at 
said moving picture data storing step 

21- A picture feature extraction method comprising: 
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a picture data identifying step for identifying 
whether the inputted picture data is of still pic- 
tures or of moving pictures; ^ 
a still picture feature extraction step including a 
resized picture generating step for generating a 
resized picture from the picture data identified 
as stiP picture data at said picture data identify- 
ing step, a frequency analyzing step for execut- *s 
ing a frequency analysis on said resized picture 
having been generated at said resized picture 
generating step, and a DCfcartial AC compo- 
nent extracting step for taWng out DC compo- 
nents and a part of AC components obtained so 
as a result of the analysis at said frequency 
analyzing step as a still picture feature data; 
and 

a moving picture feature extraction step includ- 
ing a feature data extracting frame selecting $ 5 
step for taking out some data corresponding to 
the entire frames or a portion of the frames 
from the picture data identified as moving pic- 



a picture datw storing stopfer storing sakf pic- 
ture data; and 

a picture feature data storing step using each 
picture, data stored at said picture data storing 
step for storing the still picture. feature data 
generated** said still picture: feature-extraction 
step and the* moving picture* featuntedata gen- 
erated at saidmovinirrtfliin*ffiHi^ ^ 
step as a picture feature* data 1 . 

23. A picture feature extractor method as claimed in 
daim IS, 20 or 21 , wherein- 
said resized picture generating step divides 

a still picture into blocks, end takes out from each 
block color data representative of each block. 

24. A picture feature extraction method as claimed in 
claim 1 8. 20 or 21 1 wherein 

in case when said picture data is com- 
pressed by blocks, said resized picture generating 
step takes out from each block DC components, 

25. A picture retrieving method comprising: 

a still picture feature extraction step including a 
resized picture generating stop for generating a 
resized picture from etlll picture data, a fre- 
quency analyzing step for executing' a fre- 
quency analysis on said resized picture having 
been generated at said resized picture gener- 
ating step, and a DCfcartial AC component 
extracting step for taking out DC components 
and a part of AC components obtained as a 
result of the analysis at said frequency analyz- 
ing step as a picture feature data; 
a still picture feature data correlation measur- 
ing step for calculating a correlation value 
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between the still picture feature data of the still 
picture within a still picture data base where the 
still picture feature data of the still picture data 
having been previously generated at said still 
picture feature extraction step is stored 
together with the still picture data and the still 
picture feature data generated at said still pic- 
ture feature extraction step on the basis of the 
inputted still picture data; and 
a similar still picture retrieving step for retriev- 
ing the still picture wtthin said still picture data 
base having a high correlation value with 
respect to the still picture feature data gener- 
ated on the basis of the Inputted still picture 
data as a candidate of similar stilt picture. 

26* A picture retrieving method comprising: 

a moving picture feature extraction step includ- 
ing a feature data extracting frame selecting 
step for taking out some data corresponding to 
the entire frames or a portion of the frames 
from the moving picture data, selecting the 
data subjected to processing, a resized picture 
generating step for generating a rested picture 
from data corresponding to each frame having 
been taken out a frequency analyzing step for 
executing a frequency analysts on said resized 
picture having been generated by said resized 
picture generating step, a DC/partial AC com- 
ponent extracting step for taWng out DC com- 
ponents and a part of AC components obtained 
as a result of the analysis at said frequency 
. analyzing step as a frame feature data, and a 
moving picture feature data producing step for 
producing a moving picture feature data by coK 
lecting said frame feature data outputted at 
saW DC/partial AC component extracting step; 
a moving picture feature data correlation meas- 
uring step for calculating a correlation value 
between the moving picture feature data of the 
moving picture within a moving picture data 
base where the moving picture feature data of 
the moving picture data having been previously 
generated at said moving picture feature 
extraction step is stored together with the mov- 
ing picture data and the moving picture feature 
data generated at said moving picture feature 
extraction step on the basis of the inputted 
moving picture data; and 
a similar moving picture retrieving step for 
retrieving the moving picture segment wrthin 
said moving picture data base having a high 
correlation value with respect to the moving 
picture feature data generated on the basis of 
the inputted moving picture data as a candidate 
of similar moving picture. 



27. A picture retrieving method as claimed in claim 26. 
wherein . 

said moving picture feature data correlation 
measuring step has a step of dividing the moving 
picture feature data having been generated by said 
moving picture feature extraction step on the basis 
of said inputted moving picture data, and a step of 
calculating in turn a partial correlation value 
between a part of the moving picture feature data of 
the moving picture included in said moving picture 
data base and a moving picture feature data of 
each division of the divided moving picture feature 
data, and 

said similar picture retrieving step eliminates 
the moving picture segment included in said mov- 
ing picture data base from the candidates of similar 
moving pictures when it has a feature data with a 
partial correlation value tower than a predetermined 
threshold Value- 



SO 



2& A picture retrieving method as claimed in claim 26, 
wherein 

$a)d moving picture feature data correlation 
measuring step has a step of dividing the moving 
25 picture feature data having been generated by said 
moving picture feature extraction step on the basis 
of said inputted moving picture data, and a step of 
calculating in turn a partial correlation value 
between a part of the moving picture feature data of 
30 the moving picture included in said moving picture 
data base and a partial moving picture feature data 
of each division of the divided moving picture fea- 
ture data, and 

cairi similar picture retrieving step eliminates 
3$ the moving picture segment included in said mov- 
ing picture data base from the candidates of similar 
moving pictures at the time when it is acknowl- 
edged that a number of partial moving picture fea- 
ture data having a feature data with a partial 
40 correlation value lower than a predetermined 
threshold value surpasses a predetermined thresh- 
old value. 



46 



50 



55 



29. A picture retrieving method comprising: 

a first picture feature data correlation measur- 
ing step for calculating a correlation value 
between a still picture feature data of a still pic- 
ture in inputted still picture data generated by a 
still picture feature extraction step including a 
resized picture generating step for generating a 
resized picture from the inputted picture data 
identified as still picture data, a frequency ana- 
lyzing step for executing a frequency analysis 
on said resized picture having been generated 
at said resized picture generating step, and a 
DC/partial AC component extracting step for 
taking out DC components and a part of AC 
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components obtained as a result of the analy- 
sis at said frequency analyzing step as a still 
picture feature data and a picture feature data 
of a picture within a picture data base where a 
still picture feature data having been generated 5 
at said still picture feature extraction step on 
the basis of a still picture data is stored 
together with said still picture data; 
a similar still picture retrieving step for retriev- 
ing the picture within said picture data base io 
having a high correlation value with respect to 
the picture feature data generated on the basis 
of the Inputted picture data as a candidate of 
similar still picture; 

a second picture feature data correlation meas- ts 
uring step for calculating a correlation value 
between a moving picture feature data of a 
moving picture in inputted moving picture data 
generated by a moving picture feature extrac- 
tion step including a feature data extracting so 
frame selecting step for taking cut some data 
corresponding to the entire frames or a portion 
of the frames from the picture data identified as 
moving picture data, selecting the data sub- 
jected to processing, a resized picture general- s$ 
ing step for generating a resized picture from 
data corresponding to each frame having been 
taken out, a frequency analyzing step far exe- 
cuting a frequency analysis on said resized pic- 
ture having been generated by said resized 30 
picture generating step, a DCyjsartial AC com- 
ponent extracting step for taking out DC com- 
ponents and a part of AC components obtained 
as a result of the analysis at said frequency 
analyzing step as a frame feature data, and a as 
moving picture feature data producing step for 
generating a moving picture feature data by 
collecting said frame feature data output! ad at 
said DC/|3artiaI AC component extracting step, 

and a picture feature data of a picture within a to 
picture data base where a moving picture fea- 
ture data having been generated at said mov- 
ing picture feature extraction step on the basis 
of a moving picture data is stored together with 
said moving picture data; and , 5 
a similar moving picture retrieving step for 
retrieving a moving picture segment virithin said 
picture data base having a high correlation 
value with respect to the moving picture feature 
data generated on the basis of the inputted so 
moving picture data as a candidate of similar 
moving picture. 

30. A picture retrieving method as claimed in claim 27 
wherein ' ^ 

said second picture feature data correlation 
measuring step has a step of dividing the moving 
picture feature data having been generated by said 



moving picture feature extraction step on the basis 
of said inputted moving picture data, and a step of 
calculating in turn a partial correlation value 
between a part of the moving picture feature data of 
the moving picture included in said moving picture 
data base and a moving picture feature data of 
each division of the divided moving picture feature 
data, and 

said similar picture retrieving step eliminates 
the moving picture segment Included in said mov- 
mg picture data base from the candidates of similar 
moving pictures when it has a feature data with a 
partial correlation value lower than a predetermined 
threshold value. 

31. A picture retrieving method as claimed in daim 27 
wherein 

said second picture feature data correlation 
measuring step has a step of dividing the moving 
picture feature data having been generated by said 
moving picture feature extraction step on the baa's 
of said inputted moving picture data, and a step of 
calculating in turn a partial correlation value 
between a part of the moving picture feature data of 
the moving picture- included in said moving picture 
data base and a. partial moving picture.feature dala 
erf each division af.tha cfivided moving octure, fea- 
ture data, and 

said similar picture retrieving step efiminates 
the moving picture segment included in said mov* 
ing picture data base from the candidates of similar 
moving pictures at the time when it is acknowl- 
edged that a number of partial moving picture fea- 
ture data having a feature data with a partial 
correlation value lower than a predetermined 
threshold value surpasses a predetermined thresh- 
old value. 

32. A picture retrieving method as claimed in claim 25 
28. 29, 30, or 31 , wherein 

said resized picture generating step divides 
a picture into blocks, and takes out from each block 
color information representative of each block. 

33. A picture retrieving method as claimed in claim 25 
28. 29. 30, or 3 1 . wherein 

said resized picture generating step takes 
out DC components from said each block when 
said picture data is compressed by blocks. 
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(57) In a picture feature oxtraction device, picture 
data having been determined as a moving plcturo by a 
picture data identifying section Is subjected to being tak- 
en out the data corresponding to the entire frames or a 
part of the frames by a featu re data extracting frame se- 
lecting means, upon which a resized picture generating 
means generates a resized pictu re from the data of each 
frame, and a frequency analyzing means analyzes afre- 
quency of tho resized picture having been generated. 
TTien a DC component/partial AC component extracting 
moans takes out DC components and a part of AC com- 
ponents, after which a feature extraction means gener- 
ates featuro data of the whole frames having boon se- 
lected. The picture data identified as stJN picture data is 
subjected to having a resized plcturo generated by the 
resized picture generating means from the data of each 
frame, after which the frequency analyzing moans exe- 
cutes a frequency analysis on the resized picture having 
being generated. Then as the DC component/partial AC 
component extracting means takes out DC components 
and a part of AC components obtained as a result of the 
analysis in order to generate a feature data. 
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